Modification to Fish algorithm and Integration with Web Retrieval Systems

In this paper, we first look into the drawbacks of common technologies, in which Fish algorithm is most famous, used in selective web information gathering systems. For its inherent weakness, Fish algorithm will badly waste end-user time and energy to spider valuable information. We will present a new-kind system, TH-Gatherer system. It makes some important modifications in Fish algorithm, such as taking advantage of web information retrieval system to automatically determine crawling entry-points, postponing relevance estimation step, and importing some powerful web graph algorithm. Experimental results show that this system can greatly enhance the ultimate gathering quality and in the meantime, reduce the labor of end-user to the most degree.